Papers with Measuring cross-task similarity
Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Contemporary advances in NLP are built on the representational power of latent embedding spaces learned by self-supervised language models (LMs). |
| Approach: | They use a new information theoretic probing suite to analyze representational subspaces in language models. |
| Outcome: | The proposed approach compared performance of nine tasks across 2M pre-training steps and five seeds. |